Quick & dirty get endpoints for chunks #14
Conversation
app/chunks/models.py
Outdated
| Chunk = Annotated[Union[TextChunk, ImageChunk], Field(discriminator='kind')] | ||
|
|
||
| class RetrieveResult(BaseModel): | ||
| score: Optional[float] = None |
There was a problem hiding this comment.
This feels a bit unnecessary in terms of nesting, which is what I was originlaly trying to avoid by putting the score in the chunks.
Specifically, now a user needs to do results.map(chunk: chunk.chunk.text) to get to the text.
Colud we do a middle ground? I think the other improvement (indicating the discriminator) is helpful -- so we could keep BaseChunk and still split out the Chunk = Annotated[Union[...], Field(discriminator='kind')] and then use Sequence[Chunk] in the response.
There was a problem hiding this comment.
Yeah I just ran into that updating the demo app. I'll go back to using a base type and have a per-chunktype field for extra options
There was a problem hiding this comment.
Digging into this more, I think this will need to change once the chunk/embedding data model is in place - we'll be searching for (and scoring) embeddings, and then mapping those back to chunks. I'm going to patch this up a bit for now.
kerinin
force-pushed
the
rm/get-chunks
branch
from
ad3c461
to
e7bca9c
Compare
dewy/chunks/models.py
Outdated
| """The similarity score of this chunk.""" | ||
|
|
||
| text: Optional[str] = Field(..., description="Textual description of the chunk.") | ||
|
|
||
| metadata: Union[TextChunk, ImageChunk] = Field(..., discriminator='kind') |
There was a problem hiding this comment.
This still feels oddly nested -- to get the content of the text chunk, you need to go through metadata. I think what we had here previously was cleaner -- putting things like the score into the chunk so that it is flatter.
There was a problem hiding this comment.
Yeah, this was leftover - ignore.
dewy/chunks/models.py
Outdated
|
|
||
| score: Optional[float] = None | ||
|
|
||
| class RetrieveResult(BaseModel): |
There was a problem hiding this comment.
Where is RetrieveResult used?
There was a problem hiding this comment.
doh, orphan code. removing
| document_id: int | ||
| kind: Literal["image"] = "image" | ||
|
|
||
| image: Optional[str] = Field(..., description="Image of the node.") |
There was a problem hiding this comment.
Likely some kind of location int he document? start/end index?
This makes some changes to the chunk data model to make it easier to return as a resource - I expect this will evolve further as we start actually populating the DB with chunks.
kerinin
force-pushed
the
rm/get-chunks
branch
from
c6a1249
to
6de4e32
Compare
This makes some changes to the chunk data model to make it easier to return as a resource - I expect this will evolve further as we start actually populating the DB with chunks.